…oning

Token registration is purely user-side state — it should not block on
the deployment's runtime status. Previously set_ went through
_resolve_endpoint_entry which raises 'no endpoint_url yet' when the
deployment is in DEPLOYING/PROVISIONING, dropping the user's token
along with the cache write.

Restructure set_:
1. Always fetch the deployment record (so a typo in deployment_id still
   surfaces a 404).
2. Save the token unconditionally when --token is provided.
3. Write the cache entry only when endpoint_url is already populated;
   otherwise warn that --default-model will be picked up on the first
   chat call once the deployment is READY.

The chat command's _resolve_endpoint_entry is unchanged — chat still
requires a usable endpoint to talk to.

- DeploymentChatCache/Config gain `save()` instance methods (paired with
  the existing `load()` classmethods); free functions in utils.py removed.
- `_write_text_file` writes via tmp+rename and creates the file with the
  target permission directly, closing the brief world-readable window
  that `write_text() + chmod(0600)` left open on the config file.
- `is_fresh()` flipped to `is_expired()` to align with the cache miss
  call site.
- `_resolve_endpoint_entry` had a single caller and an unused
  `default_model_override` parameter; inlined into `chat`.
- Renamed local `connection` to `connection_config` to match
  `V2ConnectionConfig`.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

…ish naming

- 401/403 from the inference endpoint now clears the stored API key for
  that deployment so the user is not silently retried with a known-bad
  token. The error message tells the user the cache was cleared.
- Replace the ad-hoc ``dict[str, Any]`` chat body with
  ``ChatCompletionRequest`` (pydantic, ``extra="allow"``) so runtime-
  variant-specific knobs supplied via ``--params`` still flow through
  while the model/messages shape is enforced.
- Rename ``chat_config_store`` → ``chat_config`` in the ``chat`` command
  and ``config`` inside the ``chat-config`` subcommands to match the
  reviewer's preferred naming and avoid shadowing the click group.
- Clarify ``_ensure_dict`` wording: payloads that are valid JSON but not
  an object now report ``non-object payload (type=...)`` instead of the
  misleading ``non-JSON response``.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

…der, named timeout consts

- Rename ``api_key`` to ``token`` across CLI flag binding, local
  variables, client method signatures, error messages, and the
  ``chat-config show`` summary label so the user-facing vocabulary
  matches the storage method names (``get_token``/``set_token``).
- Replace the length-leaking ``sk-***...***xxxx``-style mask with a
  fixed ``********`` placeholder that never reveals the token's
  prefix, suffix, or length.
- Pull ``DeploymentChatClientArgs`` magic numbers into named module
  constants (``DEFAULT_CONNECT_TIMEOUT_SEC``, ``DEFAULT_READ_TIMEOUT_SEC``).
- Update the affected test names and assertions accordingly.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

…edential convention

- Drop the bespoke tmp-and-rename / 0600-permission helper used for
  ``deployment_chat_config.json``. The existing CLI credential store
  (``client/cli/v2/config_cmd.py``) writes plain TOML without atomic
  semantics or explicit permissions; the chat config now matches that
  convention rather than introducing a stricter parallel one.
- Introduce ``write_json_file`` in ``utils.py`` so the cache and config
  models share a single, plain ``mkdir`` + ``write_text`` helper.
- Drop the ``test_config_save_enforces_0600`` test along with the
  no-longer-needed ``os``/``stat`` imports.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

…chat tests with aioresponses

- Collapse ``_read_payload``/``_ensure_dict`` into a single read-and-parse
  block inside ``_request``: parse ``resp.text()`` as JSON in one step,
  surface ``BackendAPIError`` (with the raw body in ``detail``) when the
  status is already a 4xx/5xx, and only raise ``BackendClientError``
  when a 2xx body is unparsable. The clarified comment now names
  Backend.AI's app-proxy as the layer that produces non-JSON 5xx pages.
- Remove ``--path`` from ``./bai deployment chat``. The CLI body is
  fixed to OpenAI-shaped ``{model, messages}`` via
  ``ChatCompletionRequest``, so a custom path never paired with a
  matching custom body — keeping the option encouraged the
  misconception that arbitrary inference contracts could be driven
  through this command. The SDK still accepts ``path`` as a kwarg for
  programmatic callers.
- Migrate ``test_deployment_chat_client.py`` from a real ``aiohttp.web``
  test server to ``aioresponses``-based mocks, matching the existing
  client-test convention (see ``tests/unit/client/test_resource_usage.py``).
  Headers and JSON body are asserted via ``m.requests``. New
  coverage: HTML 5xx now produces a ``BackendAPIError`` whose
  ``detail`` carries the upstream body verbatim.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

…ndAIAppProxyClient

- Add :class:`BackendAIAppProxyClient` to ``client/v2/base_client.py``:
  a ``ClientConfig``-driven base for SDK-side, direct-to-deployment
  HTTP traffic. It owns the aiohttp session, ``_request`` (with
  Bearer-token auth, app-proxy-aware JSON parsing, status-to-exception
  mapping), URL normalization, and the lifecycle hooks. The name is
  deliberately distinct from ``manager/clients/appproxy/client.py``'s
  ``AppProxyClient`` (control plane: coordinator admin API with
  ``X-BackendAI-Token``); this base sits in the SDK and handles the
  data plane (per-deployment Bearer-token traffic).
- Trim ``DeploymentChatClient`` to a single OpenAI Chat Completions
  method on top of the new base. Drop the ABC layer / separate
  ``OpenAICompatibleChatClient`` / ``DeploymentChatClientArgs`` /
  per-module timeout constants — those duties now live on
  ``BackendAIAppProxyClient`` and ``ClientConfig``. The path constant
  is renamed ``_OPENAI_COMPATIBLE_CHAT_PATH`` to make the contract
  explicit at the call site.
- Rename ``DeploymentChatAuthError`` → ``DeploymentAuthError`` since
  the 401/403 mapping now lives on the AppProxy base and is no longer
  chat-specific.
- Update the CLI to build a ``ClientConfig`` from
  ``V2ConnectionConfig`` and instantiate ``DeploymentChatClient``
  directly. Tests follow the same construction path.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

…polish help text

- ``DeploymentChatCache.remove`` → ``pop`` and
  ``DeploymentChatConfig.clear_token`` → ``pop_token`` so the names match
  the underlying ``dict.pop`` semantics (return value indicates whether
  something was actually removed).
- Inline the ``TOKEN_PLACEHOLDER`` constant into ``mask_token`` — the
  literal only has one call site.
- Reword ``./bai deployment chat-config set --token`` help text:
  "Omit when the deployment is open to public" instead of the previous
  runtime-startup phrasing.
- Update tests for the renames.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

…at/ subdirectory

Match the existing per-feature subdirectory layout used by ``./bai login``
(``~/.backend.ai/session/cookie.dat`` + ``session/config.json``):

- ``~/.backend.ai/deployment_chat.json`` →
  ``~/.backend.ai/deployment_chat/cache.json``
- ``~/.backend.ai/deployment_chat_config.json`` →
  ``~/.backend.ai/deployment_chat/config.json``

Drops the ``deployment_chat_`` filename prefix duplication and lets future
chat-related files (logs, sessions, etc.) land naturally under the same
directory.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

… omitted

Previously ``./bai deployment chat`` errored out when neither ``--model``
nor a cached ``default_model`` was provided. Now the CLI calls
``GET /v1/models`` on the deployment's inference endpoint, picks the first
``id`` (matches webui ChatCard.tsx fallback), and caches it as the
deployment's ``default_model`` so subsequent ``chat`` calls reuse it.

Add ``DeploymentChatClient.list_models()`` returning a typed
``ListModelsResponse`` so the CLI consumes ``models_response.data[0].id``
instead of dict-drilling. Hoist the ``DeploymentAuthError`` handler to the
whole ``async with`` block (auth handling is identical for both
``/v1/models`` and ``/v1/chat/completions``) and drop the per-call
``BackendAPIError`` handlers — ``_run_async`` already formats them.

…under entry

Introduce ``DeploymentChatConfigEntry { token, model }`` so per-deployment
user state lives in one nested record (mirrors ``DeploymentChatCacheEntry``)
instead of two parallel ``tokens`` / ``models`` dicts.

Resolution order in ``chat`` becomes: ``--model`` flag > ``config.model``
(user-set, ``config.json``) > ``cache.default_model`` (auto, ``cache.json``)
> ``GET /v1/models[0].id`` (auto-fetched and cached). Both fields can
co-exist; the user-set value always wins, matching the user's
"config는 사용자, cache는 자동" mental model.

CLI surface changes:
- Rename ``chat-config set --default-model`` to ``--model``; the flag now
  writes to ``config.json`` (user store) instead of ``cache.json`` (auto
  store), so the new name matches the field it sets.
- Drop the manager fetch from ``chat-config set`` — both token and model
  go to ``config.json`` only, so the command stays usable while the
  deployment is still provisioning or the manager is unreachable.
- Rename ``chat-config clear-config`` to ``chat-config clear``; clears the
  whole user config entry (token + model) for that deployment.
- Keep ``chat-config clear-cache`` for invalidating the auto-managed cache
  entry (``endpoint_url``, ``default_model``, ``last_synced_at``) on demand
  rather than waiting for the 24h TTL.
- ``chat-config show`` now prints both the user-set ``model`` and the
  auto-cached ``default_model`` so the resolved value is clear at a glance.

…es to one line

Replace the manual ``self.deployments.get(id) or DeploymentChatConfigEntry()``
+ ``self.deployments[id] = entry`` dance with a ``defaultdict``-backed store
so ``set_token`` / ``set_model`` reduce to a single bracket assignment.

Pydantic v2 cannot infer a default factory for a ``defaultdict`` whose value
is a ``BaseModel`` subclass, so the field annotation uses
``Annotated[..., Field(default_factory=...)]`` per the explicit form
``PydanticSchemaGenerationError`` directs callers to. Without it, importing
the module raises at class-construction time:

    Unable to infer a default factory for keys of type
    DeploymentChatConfigEntry. Only set, bool, str, tuple, dict, int,
    frozenset, float, list are supported, other types require an explicit
    default factory set using DefaultDict[..., Annotated[..., Field(
    default_factory=...)]]

Read paths (``get`` / ``get_token`` / ``get_model`` / ``pop_*``) still go
through ``dict.get`` / ``dict.pop`` so a missing-key lookup never plants a
stale empty entry.

…onfig

The block was restating things the code already says (method names already
imply read vs write paths) and explaining pydantic boilerplate that the
import-time error message itself points at, so it was net noise.

Relocate the wire-format and persistence Pydantic models added in this
PR into the shared `common/` tree so any backend.ai component can
consume them, not just the CLI:

- OpenAI-compat wire DTOs (`ChatCompletionMessage`,
  `ChatCompletionRequest`, `ListModelsResponse`, `ModelEntry`) →
  `common/dto/clients/openai_compat/{request,response}.py`,
  paralleling the existing `common/dto/clients/prometheus/` layout for
  third-party HTTP service contracts.
- Chat persistence data types (`DeploymentChatCache(Entry)`,
  `DeploymentChatConfig(Entry)`, `CACHE_ENTRY_TTL`) →
  `common/data/deployment_chat/types.py` as pure Pydantic models with
  no I/O coupling.

Disk load/save lives in a new
`client/cli/v2/deployment/chat/storage.py` (`load_chat_cache`,
`save_chat_cache`, `load_chat_config`, `save_chat_config`) so the data
types stay free of `client.cli` imports — `common/` MUST NOT depend on
component-specific packages per `common/dto/CLAUDE.md`. The previous
`DeploymentChatCache.load`/`.save` classmethods that pulled in
`client.cli.v2.deployment.chat.utils` are removed in favor of these
free functions, eliminating the backward dependency.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

…e in

Reverse the relocation done in 69fd070:

- `DeploymentChatCache(Entry)` and `DeploymentChatConfig(Entry)` (and
  `CACHE_ENTRY_TTL`) move from `common/data/deployment_chat/` back to
  `client/cli/v2/deployment/chat/types.py`.
- `load_chat_*` / `save_chat_*` free functions go away; the
  corresponding `.load()` / `.save()` classmethods are restored on
  `DeploymentChatCache` / `DeploymentChatConfig`.
- `client/cli/v2/deployment/chat/storage.py` is removed — the typed
  models own their own disk format directly.

`common/dto/clients/openai_compat/{request,response}.py` (the
OpenAI-compat wire DTOs) are left in place, since those are reused by
the SDK and may grow more component consumers.

Address review feedback on PR #11344 — the OpenAI-compat chat endpoint
treats each turn as a "message" with role/content, so the user-facing
CLI argument is more naturally named `message`. Update the click
argument declaration, the function parameter, the help text, and the
request-body construction site.

The JSON key on the wire stays `content` (that's the OpenAI spec); only
the local variable / argument name changes.

…onfig

`chat-config show` was printing both the cache (auto-managed
`endpoint_url` / `default_model` / `last_synced_at`) and the user's
config (`token` / `model`) in one block, which blurred the
responsibility split between the two files.

Trim the command to print only the config entry it owns. Drop
``DeploymentChatFormatter.print_summary``/``entry_lines`` (the only
consumers of the cache half) in favor of a dedicated
``print_config(deployment_id, entry)``. Update the formatter test to
match.

…mand group

The auto-managed cache and the user-managed config are two separate
files (``cache.json`` vs ``config.json``); having a `clear-cache`
subcommand under `chat-config` mixed the two responsibilities.

Replace ``./bai deployment chat-config clear-cache`` with a dedicated
``chat-cache`` group:

- ``./bai deployment chat-cache show <id>`` — print the cached
  endpoint metadata (``endpoint_url``, ``default_model``,
  ``last_synced_at``) for inspection / debugging.
- ``./bai deployment chat-cache clear <id>`` — drop the cache entry,
  forcing the next ``chat`` call to refetch endpoint and re-derive the
  default model.

``DeploymentChatFormatter`` gains ``print_cache(deployment_id, entry)``
to render the cache view; the `chat-config clear` docstring is updated
to reference the new path.

…args

Address review feedback (PR #11412 discussion r3165318334) — the
deployment id values flowing through the chat data classes, the
formatter, and the click handlers represent a deployment, not a generic
UUID. Switch the static signatures to
``ai.backend.common.identifier.deployment.DeploymentID`` (a
``NewType(UUID)``) so type checkers can distinguish deployment ids
from other UUIDs without any runtime cost.

The click ``type=click.UUID`` parser still emits a plain ``UUID`` at
runtime; ``DeploymentID`` is structurally identical, so the wrap is
implicit and no conversion is needed at the boundary.

…context (#11412)

…history None vs empty

- Extract `_OpenAICompatModel` base class so all OpenAI-compat response DTOs
  share a single `ConfigDict(extra="allow")` declaration instead of repeating
  it on each subclass.
- In `history_show`, distinguish "no history record" (`messages is None`) from
  the invariant-violating "record exists but empty list" case so the CLI
  message reflects the actual state instead of conflating both as falsy.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

Python's `pop` convention (`dict.pop`, `list.pop`) implies the popped value
is returned, but these methods return a plain `bool` because every caller
only needs "did anything actually get removed?" Renaming so the method
names match what the calls do:

- `DeploymentChatCache.pop`        → `delete`        (removes the entry)
- `DeploymentChatConfig.pop`       → `delete`        (removes the entry)
- `DeploymentChatConfig.pop_token` → `clear_token`   (nulls the field, drops the entry only when both fields are unset)
- `DeploymentChatConfig.pop_model` → `clear_model`   (same shape as `clear_token`)

`pop_token`/`pop_model` were already misnomers — they null one field rather
than fully popping the entry, so `clear_*` reflects the actual behavior.
Return types stay `bool` since no caller uses the popped value.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

…lient._request`

Address PR #11344 review: split JSON parsing and payload validation
into a dedicated method so `_request` only orchestrates the HTTP call
and status handling.

`PrometheusQueryPresetRepository.preview_template` was rewired to call
`PrometheusClient.preview_query_template` in #11274, but the component
test added in #11482 still mocked the now-unused `query_instant`. The
real client method falls through and returns an `AsyncMock`, so the
PrometheusResponse model fails validation and the API returns 500.

Mock the method actually called so the preview-endpoint tests cover
the success path and the FailedToGetMetric → PrometheusQueryEvaluationFailed
mapping again.